Image capturing apparatus, control method thereof and storage medium storing control program therefor

ABSTRACT

In the image capturing apparatus, the controller controls an optical element, when a motion of the image capturing apparatus follows a motion of the object, by using first motion information obtained from a first detector to detect the motion of the image capturing apparatus and second motion information obtained from a second detector to detect the motion of the object. The calculator calculates prediction information on the motion of the object during an exposure time, by using the second information detected at multiple times before the exposure time. The controller uses the prediction information to control the optical element during the exposure time.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of application Ser. No. 15/056,779,filed Feb. 29, 2016 the entire disclosure of which is herebyincorporated by reference.

FIELD OF THE INVENTION

The present invention relates to a technique of reducing image blur in aso-called “follow shot”.

DESCRIPTION OF THE RELATED ART

The follow shot, which enables expressing a sense of speed of a movingobject, is a technique of photography to acquire a captured image inwhich the object is still and a background flows, by panning an imagecapturing apparatus (camera) so as to follow the motion of the object.In such follow shot, a faster or slower panning speed than a motionvelocity of the object generates a captured image including a blurredobject image.

Japanese Patent Laid-Open No. 04-163535 discloses a camera that correctssuch object image blur by moving part of an optical system or an imagesensor during image capturing (exposure) on a basis of an angularvelocity of the object relative to the camera calculated before theimage capturing and an angular velocity of the camera during the imagecapturing obtained from an angular velocity sensor. This cameracalculates the angular velocity relative to the camera (hereinafterreferred to as “a relative object angular velocity”) by using an outputfrom the angular velocity sensor and a displacement amount of the objectimage on an image plane; the displacement amount is detected fromtemporally sequential captured images.

The camera disclosed in Japanese Patent Laid-Open No. 04-163535 premisesthat the relative object angular velocity is uniformly maintained duringthe image capturing in which the image blur is corrected. However, evenif the moving object (for example, a train) is in uniform linear motion,the relative object angular velocity measured from the camera located ina direction orthogonal to a motion direction of the object changes(increases or decreases). In this case, when a measurement time of therelative object angular velocity and a time of actual image capturinghave a time lag, disregarding the change of the relative object angularvelocity during the time lag makes it impossible to adequately correctthe image blur during the image capturing.

BRIEF SUMMARY OF THE INVENTION

The present invention provides an image capturing apparatus capable ofperforming a good follow shot with reduced object image blur even whenan object velocity detected from a camera changes.

The present invention provides as an image capturing apparatusconfigured to perform image capturing of an object. The apparatusincludes a controller configured to control an optical element, when amotion of the image capturing apparatus follows a motion of the object,by using first motion information obtained from a first detector todetect the motion of the image capturing apparatus and second motioninformation obtained from a second detector to detect the motion of theobject. The calculator configured to calculate prediction information onthe motion of the object during an exposure time, by using the secondinformation detected at multiple times before the exposure time. Thecontroller is configured to use the prediction information to controlthe optical element during the exposure time.

The present invention provides as another aspect thereof an imagecapturing apparatus configured to perform image capturing of an object.The apparatus includes a controller configured to control an opticalelement, when a motion of the image capturing apparatus follows a motionof the object, by using first motion information obtained from a firstdetector to detect the motion of the image capturing apparatus andsecond motion information obtained from a second detector to detect themotion of the object, and a calculator configured to calculateprediction information on the motion of the object during an exposuretime, by using the second information detected at multiple times beforethe exposure time. The controller is configured to use the predictioninformation and the first motion information obtained during theexposure time to control the optical element during the exposure time.

The present invention provides as still another aspect thereof a method,a computer program or a non-transitory computer-readable storage mediumstoring the control program to control an optical element as above in animage capturing apparatus.

Other aspects of the present invention will be apparent from theembodiments described below with reference to the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart showing an angular velocity setting process in acamera that is Embodiment 1 of the present invention.

FIG. 2 is a flowchart showing a follow shot assist process in the cameraof Embodiment 1.

FIG. 3 is a block diagram showing a configuration of the camera ofEmbodiment 1.

FIG. 4 is a block diagram showing a configuration of an imagestabilizing system in the camera of Embodiment 1.

FIG. 5 is a flowchart showing a panning control in the camera ofEmbodiment 1.

FIG. 6 is a block diagram showing a configuration of a shift drivecontrol system in a follow shot assist mode in the camera of Embodiment1.

FIG. 7 shows panning determination thresholds in the camera ofEmbodiment 1.

FIG. 8 is a graph showing a relative object angular velocity and itschange (angular acceleration) in Embodiment 1.

FIG. 9 shows the relative object angular velocity in Embodiment 1.

FIG. 10 shows singular points in Embodiment 1.

FIG. 11 shows a determination of a 0°-singular point in Embodiment 1.

FIG. 12 is a flowchart showing an angular velocity setting process in acamera of Embodiment 2 of the present invention.

FIGS. 13A to 13D each show a distance between two points and an angletherebetween in Embodiment 2.

FIG. 14 is a flowchart showing a follow shot assist process in thecamera of Embodiment 3.

FIG. 15 is a block diagram showing a configuration of alens-interchangeable camera system that is Embodiment 4 of the presentinvention.

FIG. 16 is a block diagram showing a configuration of a follow shotassist control system in an interchangeable lens of Embodiment 4.

FIG. 17 is a flowchart showing a camera-side follow shot assist processin Embodiment 4.

FIG. 18 is a flowchart showing a lens-side follow shot assist process inEmbodiment 4.

FIG. 19 is a flowchart showing a camera-side follow shot assist processin a lens-interchangeable camera system that is Embodiment 5 of thepresent invention.

FIG. 20 is a flowchart showing an interchangeable lens-side follow shotassist process in Embodiment 5.

FIG. 21 is a flowchart showing a camera-side follow shot assist processin a modified example of Embodiment 5.

FIG. 22 is a flowchart showing a lens-side follow shot assist process inthe modified example of Embodiment 5.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described belowwith reference to the accompanied drawings.

Embodiment 1

FIG. 3 shows a configuration of a lens-interchangeable camera(hereinafter simply referred to as “a camera”) 100 as an image capturingapparatus that is a first embodiment (Embodiment 1) of the presentinvention.

The camera 100 is provided with an image capturing lens unit 101 that isan image capturing optical system causing light from an object to forman optical image (object image). The image capturing lens unit 101includes a main lens system 102, a zoom lens 103 that is movable in anoptical axis direction in which an optical axis of the image capturinglens unit 101 extends to vary a focal length of the image capturing lensunit 101, and a focus lens (not shown) that is movable in the opticalaxis direction to perform focusing. The image capturing lens unit 101further includes a shift lens 104 that is an optical elementconstituting part thereof.

The shift lens 104 is a shift element that is movable (or shiftable) indirections orthogonal to the optical axis (hereinafter correctivelyreferred to as “a shift direction”) to perform a follow shot assist. Thefollow shot assist is performed, in a follow shot to capture an objectimage of a moving object while a user changes a direction of the camera100 by panning thereof, to reduce blur of the object image. The shiftlens 104 also has an image stabilizing function to optically correctblur of the object image due to a shake of the camera 100 caused byuser's hand jiggling (the shake is hereinafter referred to as “a camerashake”), by shifting the shift lens 104 in the directions orthogonal tothe optical axis.

The camera 100 is provided with a zoom encoder 105, a shift positionsensor 106, an angular velocity sensor 107, an angular velocityamplifier 108, a camera control microcomputer 130, a shift driver 109and a shift position amplifier 110.

The zoom encoder 105 detects a position of the zoom lens 103 in theoptical axis direction. The shift position sensor 106 detects a positionof the shift lens 104 in the shift direction. The angular velocitysensor 107 as a first detector detects an angular velocity (angularvelocity information) that is a motion velocity of the camera 100 indirections (pitch and yaw directions) orthogonal to the optical axis.The angular velocity amplifier 108 amplifies an output from the angularvelocity sensor 107.

The camera control microcomputer (hereinafter simply referred to as “acamera microcomputer”) 130 controls operations of the entire camera 100.The shift driver 109 includes a shift actuator such as a voice coilmotor and its driver circuit and shifts the shift lens 104 by drivingthe shift actuator. The shift position amplifier 110 amplifies an outputfrom the shift position sensor 106.

The camera 100 is further provided with a shutter 111, an image sensor112, an analog signal processing circuit 113, a camera signal processingcircuit 114, a timing generator 115, operation switches 116, a shuttermotor 117 and a shutter driver 118.

The image sensor 112 is constituted by a photoelectric conversionelement such as a CMOS sensor or a CCD sensor and photoelectricallyconverts the object image formed by the image capturing lens unit 101 tooutput an analog electric signal. The shutter 111 controls an exposuretime (in other words, a length of time of an exposure) of the imagesensor 112.

The analog signal processing circuit (AFE) 113 amplifies the analogsignal output from the image sensor 112 and converts the amplifiedanalog signal into an image capturing signal as a digital signal tooutput it to the camera signal processing circuit 114.

The camera signal processing circuit 114 produces a video signal(captured video image) by performing various image processing on theimage capturing signal. The captured video (or a captured still imageextracted therefrom) is recorded to a memory card 119 that is detachablyattached to the camera 100 or is displayed on a monitor (hereinafterreferred to as “an LCD”) 120 constituted by a display element such as aliquid crystal panel.

The timing generator 115 sets operation times of the image sensor 112and the analog signal processing circuit 113.

The operation switches 116 include various switches such as a powerswitch, a release switch and a mode selection switch, and dials. Thecamera 100 in this embodiment is switchable between a follow shot assistmode and a normal image capturing mode through the operation of the modeselection switch. The shutter motor 117 is driven by the shutter driver118 to cause the shutter 111 to perform a charging operation (closingoperation).

The camera signal processing circuit 114 includes a motion vectordetector 135 as a second detector that detects a motion vector fromframe images constituting the captured video image.

The camera microcomputer 130 further includes an image stabilizationcontroller 131, a follow shot controller 132, a shutter controller 133and an object angular velocity calculator 134. The object angularvelocity calculator 134 corresponds to a calculator, and the follow shotcontroller 132 corresponds to a controller.

The image stabilization controller 131 performs an image blur correctioncontrol (image stabilization control) to control shift drive of theshift lens 104 so as to correct (reduce) the blur of the object image,in other words, image blur due to the camera shake.

The follow shot controller 132 controls the shift drive of the shiftlens 104 to perform the follow shot assist.

The shutter controller 133 stops energization of a releaseelectromagnetic magnet (not shown) through the shutter driver 118 tocause the shutter 111 to perform an opening operation from its chargedstate and controls the shutter motor 117 to cause the shutter 111 toperform the charging operation.

The object angular velocity calculator 134 calculates a relative objectangular velocity as a measured angular velocity of the object (mainobject) with respect to the camera 100. The main object means an imagecapturing target. The camera microcomputer 130 performs focus lenscontrol, aperture stop control and others.

In response to an ON-operation of the power switch in the operationswitches 116 to turn power of the camera 100 on, the cameramicrocomputer 130 starts power supply to each of the above-describedparts in the camera 100 and performs a necessary initial setting.

In the normal image capturing mode that is not the follow shot assistmode, the angular velocity sensor 107 detects the camera shake, and theimage stabilization controller 131 shifts the shift lens 104 dependingon a result of the detection to correct the image blur due to the camerashake.

FIG. 4 shows a configuration of the image stabilizing system of thecamera 100. In FIG. 4, components common to those in FIG. 3 are denotedby the same reference numerals as those in FIG. 3, and descriptionthereof is omitted. Although an actual image stabilizing system has twosystems shifting the shift lens 104 in two directions (pitch and yawdirections), FIG. 4 shows one thereof since configurations thereof areidentical to each other.

An angular velocity A/D converter 401 converts an angular velocitysignal (analog signal) output from the angular velocity sensor 107(angular velocity amplifier 108) into angular velocity data as a digitalsignal to output it to a filter calculator 402. The angular velocitydata is sampled at a frequency of about 1-10 kHz corresponding to afrequency of the camera shake.

The filter calculator 402, which is constituted by a high-pass filter(HPF), removes an offset component contained in the angular velocitydata and changes a cutoff frequency of the HPF in response to aninstruction from a panning controller 407 described below. A firstintegrator 403 converts the angular velocity data into angulardisplacement data, in order to produce target position data that is dataof a target shift position of the shift lens 104.

A shift position A/D convertor 406 converts a shift position signal(analog signal) output from the shift position sensor 106 (shiftposition amplifier 110) into shift position data as a digital signal. Afirst adder 404 subtracts the shift position data (current shiftposition data) from the target position data of the shift lens 104 tocalculate drive amount data of the shift lens 104.

A PWM outputter 405 outputs the calculated drive amount data to theshift driver 109. The shift driver 109 drives the shift actuator on abasis of the drive amount data to shift the shift lens 104 to the targetshift position.

The panning controller 407 determines, from the angular velocity dataobtained from the angular velocity sensor 107 (angular velocity A/Dconvertor 401), whether or not panning of the camera 100 is beingperformed. If it is determined that the panning of the camera 100 isbeing performed, the panning controller 407 changes the cutoff frequencyof the filter calculator (HPF) 402 and adjusts the output of the firstintegrator 403.

FIG. 5 shows an example of a panning control performed by the panningcontroller 407. The panning controller 407 (that is, the cameramicrocomputer 130) performs this panning control according to a panningcontrol program as a computer program.

At step S501, the panning controller 407 determines whether or not anaverage value of the angular velocity data taken in from the angularvelocity A/D convertor 401 is larger than a predetermined value a. Theaverage value (hereinafter referred to as “an angular velocity averagevalue”) is an average value of the angular velocity data sampled apredetermined number of times. If the angular velocity average value isequal to or lower than the predetermined value a, the panning controller407 determines that the panning is not being performed and proceeds tostep S507. On the other hand, when the angular velocity average value islarger than the predetermined value a, the panning controller 407proceeds to step S502 to determine whether or not the angular velocityaverage value is larger than a predetermined value b (>a). If theangular velocity average value is equal to or lower than thepredetermined value b, the panning controller 407 determines that a slowpanning is being performed and proceeds to step S506. If the angularvelocity average value is larger than the predetermined value b, thepanning controller 407 determines that a fast panning is being performedand proceeds to step S503.

At step S503, the panning controller 407 sets the cutoff frequency ofthe filter calculator (HPF) 402 to a maximum value. Next at step S504,the panning controller 407 turns the image stabilization control off(that is, to a non-operation state). A reason for tuning the imagestabilization control off when the fast panning is being performed isthat shifting the shift lens 104 by regarding the fast panning as alarge camera shake moves the captured image noticeably when the shiftlens 104 reaches its shift end, which provides a feeling of strangenessto a user. Another reason therefor is that the fast panning moves thecaptured image largely and therefore the image blur due to the camerashake provides little feeling of strangeness to the user. Furthermore,gradually stopping the shift of the shift lens 104 after setting thecutoff frequency of the HPF to the maximum value enables preventing theimage blur due to the camera shake from abruptly appearing in responseto the turning off of the image stabilization control and thus providinga feeling of strangeness to the user.

The panning controller 407 having turned the image stabilization controloff gradually changes at step S505 the output of the first integrator403 from the current angular displacement data to initial position data.This gradual change of the output of the first integrator 403 graduallyreturns the shift lens 104 to its initial position where an optical axisof the shift lens 104 coincides with the optical axis of the imagecapturing lens unit 101.

The panning controller 407 having determined that the slow panning isbeing performed sets at step S506 the cutoff frequency of the filtercalculator (HPF) 402 depending on the angular velocity data. This isbecause the image blur due to the camera shake is likely to benoticeable during the slow panning, and such image blur is necessary tobe corrected. The cutoff frequency is set such that the image blur dueto the camera shake can be corrected while an unnatural change of thecaptured image is prevented during the panning. Then, at step S508, thepanning controller 407 turns the image stabilization control on (thatis, to an operation state).

The panning controller 407 having determined that the angular velocityaverage value is equal to or lower than the predetermined value a (thatis, that the panning is not being performed) and thus proceeded to stepS507 sets the cutoff frequency of the filter calculator (HPF) 402 to anormal value. Then, the panning controller 407 proceeds to step S508 toturn the image stabilization control on.

FIG. 7 shows a relation between the angular velocity data in the yawdirection during the panning and the predetermined values a and b.Reference numeral 701 in FIG. 7 denotes the angular velocity datasampled. The angular velocity data has a positive (+) value when arightward panning of the camera 100 is performed and has a negative (−)value when a leftward panning thereof is performed. In FIG. 7, arightward fast (steep) panning, a rightward slow panning and a leftwardslow panning are detected.

As shown in FIG. 7, the angular velocity data significantly deviatesfrom its initial value (0) during the panning. The output of the firstintegrator 403 integrating this angular velocity data to calculate thetarget position data of the shift lens 104 extremely increases due to aDC-like offset component, which makes the shift lens 104 uncontrollable.Therefore, when the panning is detected, it is necessary to set thecutoff frequency of the HPF to be high so as to cut the offsetcomponent.

In particular, when the fast panning is being performed, such anuncontrollable state is likely to appear, so that it is necessary to setthe cutoff frequency of the HPF to be high so as to prevent the outputof the first integrator 403 from increasing.

The panning control described above enables, even during the panning,producing a captured image providing little sense of strangeness to theuser. In FIG. 3, in response to setting of the follow shot assist modeby the operation of the mode selection switch in the operation switches116, the motion vector detector 135 in the camera signal-processingcircuit 114 detects the motion vector of the object image fromsequential frame images. The detected motion vector is input to thefollow shot controller 132 in the camera microcomputer 130. Alongtherewith, the follow shot controller 132 receives the angular velocitysignal (first motion information) from the angular velocity sensor 107(angular velocity amplifier 108).

The motion vectors output from the motion vector detector 135 during thefollow shot include a motion vector of a main object image that is acapturing target image and a motion vector of a background image flowingin the back of the main object image. One of these motion vectors whichshows a smaller motion amount than that shown by the other is the motionvector of the main object image. This motion vector (second motioninformation) of the main object image shows a displacement (motion) ofthe main object image on an image plane, that is, on the image sensor112 during one frame period.

On the other hand, the angular velocity data output from the angularvelocity sensor 107 corresponds to a panning velocity (follow shotvelocity) of the camera 100. Calculating a difference between thisangular velocity data and an angular velocity calculated from an amountof the displacement of the main object image on the image plane duringthe one frame period and a focal length of the image capturing lens unit101 provides an angular velocity of the main object relative to thecamera 100 (that is, the relative object angular velocity).

The object angular velocity calculator 134 calculates (acquires) therelative object angular velocity at each time at which the frame imageis produced, that is, at a frame period. The object angular velocitycalculator 134 sends, to the follow shot controller 132, information ona set of the calculated relative object angular velocity and acalculation time (acquisition time) at which the relative object angularvelocity was calculated.

FIG. 6 shows a configuration of a shift drive control system thatperforms a shift drive control of the shift lens 104 in the follow shotassist mode. In FIG. 6, components common to those in FIGS. 3 and 4 aredenoted by the same reference numerals as those in FIGS. 3 and 4, anddescription thereof is omitted.

The follow shot controller 132 includes a camera information acquirer601, an angular velocity data outputter 602, an object angular velocitysetter 603, a second adder 604, a second integrator 605 and a settingchanger 606.

The camera information acquirer 601 acquires, from the operationswitches 116, follow shot setting information showing that the followshot assist mode is set by the operation of the mode selection switchand release information showing that image capturing is instructed by anoperation of the release switch. The angular velocity data outputter 602samples the angular velocity data at predetermined times and outputs thesampled data to the object angular velocity calculator 134.

The object angular velocity setter 603 acquires information on the set(multiple sets) of the relative object angular velocity calculated bythe object angular velocity calculator 134 before image capturing forrecording (that is, before the exposure of the image sensor 112 forrecording a captured still image) and its calculation time. The objectangular velocity setter 603 holds (accumulates) the acquired informationas an angular velocity history. In the following description, theexposure means the image capturing for recording. The object angularvelocity setter 603 acquires a relative object angular velocity relativeto the camera 100 as a predicted object angular velocity (predictioninformation) during the exposure time, by calculation or the like usingthe angular velocity history before the exposure. The relative objectangular velocity calculated by the object angular velocity calculator134 before the exposure is hereinafter referred to as “a pre-exposurerelative object angular velocity”, the angular velocity history beforethe exposure is hereinafter referred to as “a pre-exposure angularvelocity history”, and the relative object angular velocity during theexposure time is hereinafter referred to as “an in-exposure relativeobject angular velocity”. The object angular velocity setter 603 setsthe acquired in-exposure relative object angular velocity as a relativeobject angular velocity to be used for control of the shift dive of theshift lens 104 during the exposure time in the follow shot assist.

The second adder 604 calculates a difference between the angularvelocity data from the angular velocity sensor 107 and the in-exposurerelative object angular velocity set by the object angular velocitysetter 603. The second integrator 605 performs an integral operationonly during the exposure time. The setting changer 606 changes thesetting of the panning controller 407 in response to a notice ofacquisition of the follow shot setting information from the camerainformation acquirer 601. In response to setting of the follow shotassist mode by the operation of the mode select switch in the operationswitches 116, the camera information acquirer 601 notifies the followshot setting information to the setting changer 606. The setting changer606 changes, in response to the notice of the follow shot settinginformation, the predetermined values a and b in the panning controller407 in order not to limit a fast panning by the user.

Furthermore, the second adder 604 calculates the difference between theangular velocity data from the angular velocity sensor 107 and thein-exposure relative object angular velocity from the object angularvelocity setter 603, and sends the difference to the second integrator605.

The second integrator 605 starts, in response to the release informationfrom the camera information acquirer 601, the integration operation ofthe above difference during the exposure time and outputs its result.The second integrator 605 outputs a value by which the shift lens 104 islocated at its initial position at a time other than the exposure time.There is no problem in a shift of the shift lens 104 at the end of theexposure time from its position thereat to the initial position in ashort time. That is, the analog signal from the image sensor 112 is readout immediately after the end of the exposure time and therefore the LCD120 does not display the captured image, so that a motion of thecaptured image due to the shift of the shift lens 104 does not become aproblem.

The output of the second integrator 605 is added to the output of thefirst integrator 403 by the first adder 404. Then, from the additionresult, the shift position data of the shift lens 104 from the shiftposition sensor 106 (shift position A/D converter 406) is subtracted,thereby calculating the drive amount data of the shift lens 104.

In the follow shot assist mode, when the fast panning is actuallyperformed by the user, the panning controller 407 immediately starts thepanning control and turns the image stabilization control off asdescribed at step S504 in FIG. 5. The shift lens 104 subjected to thepanning control corrects the displacement amount of the object image onthe image plane; the displacement amount corresponds to a differencebetween the angular velocity of the panning of the camera 100 and therelative object angular velocity that is the angular velocity of themain object (hereinafter simply referred to as “an object”) relative tothe camera 100. With this panning control, a difference, which causes anunsuccessful follow shot, between the panning velocity of the camera 100and a motion velocity of the object during the exposure time iscancelled out by the shift drive of the shift lens 104, which results ina successful follow shot.

The object angular velocity setter 603 takes a release time lag and theexposure time into consideration, when setting the in-exposure relativeobject angular velocity by using the pre-exposure angular velocityhistory obtained from the object angular velocity calculator 134 andaccumulated before the exposure.

For example, when the follow shot is performed for an object in uniformlinear motion with the camera 100 located in a direction orthogonal to amotion direction of the object, the angular velocity measured from thecamera 100 changes continuously. For this reason, the measured angularvelocity of the object and an actual angular velocity thereof during theexposure time do not become equal to each other. Accordingly, leavingthis change of the angular velocity (that is, an angular acceleration)out of consideration makes it impossible to achieve a sufficientcorrection by the shift drive of the shift lens 104.

FIG. 8 shows a change of an angular velocity co of an object (train) inuniform linear motion as shown in FIG. 9. The angular velocity ω ismeasured from the camera 100 located in the direction orthogonal to themotion direction of the object. In FIG. 9, the object is in the uniformlinear motion leftward at a velocity v. A point (hereinafter referred toas “an origin”) A shows a position where a distance from the camera 100to the object becomes shortest on a motion track of the object in theuniform linear motion. L denotes the distance from the camera 100 to theorigin A (that is, the shortest distance from the camera 100 to themotion track). In addition, θ denotes an angle formed by a directionfrom the camera 100 to the object (that is, a direction of the camera100) with respect to a direction from the camera 100 to the origin A, inother words, with respect to the direction orthogonal to the motiondirection of the object. The angle θ is hereinafter referred to as “apanning angle”. The panning angle θ has a positive (+) value on a rightside further than the origin A and has a negative (−) value on a leftside further than the origin A.

In FIG. 8, a horizontal axis shows the panning angle θ that becomes 0when the object in FIG. 9 is located at the origin A, and a centralvertical axis shows the angular velocity ω of the object. A solid lineshows a change of the angular velocity ω. Furthermore, a right verticalaxis shows an angular acceleration α, and a dashed line graph shows achange of the angular acceleration α.

The change of the angular acceleration α herein is a change of anangular acceleration of the object depending on the position thereofrelative to the position of the camera 100. FIG. 8 shows an example ofthe angular velocity ω and the angular acceleration α in a case wherethe shortest distance from the camera 100 to the origin A is 20 m andthe object is in uniform linear motion at a velocity of 60 km/h.

In FIG. 8, when the object passes the origin A (θ=0°), the angularvelocity ω becomes maximum and the angular acceleration α becomes 0.When the object passes a position of θ=+30°, the angular acceleration αbecomes maximum. When the object passes a position of θ=−30°, theangular acceleration α becomes minimum. This relation between thepanning angle θ and the angular velocity ω and angular acceleration αdoes not depend on the above-described shortest distance and the motionvelocity of the object.

FIG. 2 is a flowchart showing a follow shot assist process performed bythe camera microcomputer 130 in the follow shot assist mode. The cameramicrocomputer 130 executes this process according to a follow shotassist control program that is a computer program. The user performspanning of the camera 100 to follow a moving object.

At step S201, the camera microcomputer 130 determines whether or not ahalf-press operation (SW1ON) of the release switch is performed. If theSW1ON is performed, the camera microcomputer 130 proceeds to step S202to increment a time measurement counter and then proceeds to step S204.If the SW1ON is not performed, the camera microcomputer 130 proceeds tostep S203 to reset the time measurement counter and then returns to stepS201.

At step S204, the camera microcomputer 130 checks whether or not thepre-exposure relative object angular velocity (in FIG. 2 simply writtenas “pre-exposure object angular velocity”) has been already calculatedby the object angular velocity calculator 134. When the pre-exposurerelative object angular velocity has been already calculated, the cameramicrocomputer 130 proceeds to step S205 to check whether or not the timemeasurement counter has reached a predetermined time T. If thepre-exposure relative object angular velocity has not been yetcalculated or the time measurement counter has reached the predeterminedtime T even though the pre-exposure relative object angular velocity hasbeen already calculated (that is, a time during the SW1ON is performedis longer than the predetermined time T), the camera microcomputer 130proceeds to step S206.

At step s206, the camera microcomputer 130 causes the object angularvelocity calculator 134 to calculate the pre-exposure relative objectangular velocity. This process as a first process causes the objectangular velocity calculator 134 to calculate the relative object angularvelocity before the exposure started in response to SW2ON mentionedbelow and causes the object angular velocity setter 603 to acquire thepre-exposure angular velocity history.

A reason for recalculating the pre-exposure relative object angularvelocity when the time measurement counter has reached the predeterminedperiod T is to take into consideration a possibility that the objectvelocity changes within the predetermined time T. The pre-exposurerelative object angular velocity calculated by the object angularvelocity calculator 134 is sent, at each calculation thereof, to theobject angular velocity setter 603 in the follow shot controller 132. Ifthe time measurement counter has not yet reached the predeterminedperiod T at step S205, the camera microcomputer 130 proceeds to stepS208.

At step S207 after step S206, the camera microcomputer 130 causes theobject angular velocity setter 603 to set the in-exposure relativeobject angular velocity (in FIG. 2 simply written as “in-exposure objectangular velocity”). This process (angular velocity setting process) as asecond process will be described in detail below. The cameramicrocomputer 130 then proceeds to step S208.

At step S208, the camera microcomputer 130 determines whether or not afull-press operation (SW2ON) of the release switch is performed. If theSW2ON is not performed, the camera microcomputer 130 returns to stepS201. On the other hand, if the SW2ON is performed, the cameramicrocomputer 130 proceeds to step S209 to start the exposure by causingthe shutter 111 to open through the shutter controller 133.

Furthermore, at step S210, the camera microcomputer 130 causes thefollow shot controller 132 to control the shift drive of the shift lens104 depending on the in-exposure relative object angular velocity set atstep S207, thereby performing the follow shot assist to correct thedisplacement amount of the object image on the image plane. In thiscontrol of the shift drive, if a determining that the fast panning isbeing performed is made at step S502 in FIG. 5, the camera microcomputer130 performs the shift drive of the shift lens 104 in order to correctthe image blur due to the camera shake through the image stabilizationcontroller 131.

Next, at step S211 the camera microcomputer 130 determines whether ornot the exposure has completed. If the exposure has completed, thecamera microcomputer 130 proceeds to step S212. If the exposure has notyet completed, the camera microcomputer 130 returns to step S210.

At step S212, the camera microcomputer 130 determines again whether ornot the SW2ON is performed. If the SW2ON is performed, the cameramicrocomputer 130 returns to step S209 to perform a next exposure (thatis, to perform image capturing of a next image in continuous shot). Onthe other hand, if the SW2ON is not performed, the camera microcomputer130 returns to step S201.

FIG. 1 is a flowchart showing the angular velocity setting processperformed by the object angular velocity setter 603 at step S207 in FIG.2. The object angular velocity setter 603 (that is, the cameramicrocomputer 130) executes this process according to part of the followshot assist control program.

At step S101, the object angular velocity setter 603 receiving aninstruction of setting the in-exposure relative object angular velocityfrom the camera microcomputer 130 reads out the pre-exposure angularvelocity history that the object angular velocity setter 603 haspreviously acquired from the object angular velocity calculator 134 andaccumulated.

Then, at step S102, the object angular velocity setter 603 detects, fromthe multiple sets of the pre-exposure relative object angular velocityand the calculation time contained in the read-out pre-exposure angularvelocity history, a singular point of the angular velocity. In FIG. 8,at three singular points of the angular velocity where the panningangles θ are 0°, +30° and −30°, three types of specific changes in theangular acceleration that is a temporal change rate of the angularvelocity are generated.

The object angular velocity setter 603 calculates the angularacceleration (angular acceleration information) by dividing a differenceof the relative object angular velocities at two mutually adjacentcalculation times in the pre-exposure angular velocity history by a timeinterval between the two mutually adjacent calculation times. The objectangular velocity setter 603 performs this angular accelerationcalculation for multiple sets of the two mutually adjacent calculationtimes to calculate temporal changes of the angular acceleration. Then,the object angular velocity setter 603 performs a process to detect, inthe angular acceleration that temporally changes, a positive localmaximum value (that is, a change from increase to decrease at θ=+30°)and a negative local maximum value (that is, a change from decrease toincrease at θ=−30°). The object angular velocity setter 603 alsodetects, in the angular acceleration, a change between positive andnegative (that is, a change from one of positive and negative to theother at θ=0°).

At step S103, the object angular velocity setter 603 determines whetheror not the number of the singular points detected by the above-describedsingular point detection process is two or more. That is, the objectangular velocity setter 603 determines whether or not the singularpoints corresponding to, among the panning angles θ=0°, +30° and −30°,“0°, +30° and −30°”, “+30° and 0°” or “0° and −30°” are detected. If thetwo or more singular points are detected, the object angular velocitysetter 603 proceeds to step S104, and otherwise proceeds to step S105.

At step S104, the object angular velocity setter 603 calculates (sets)the in-exposure relative object angular velocity ω by using followingexpression (1)

$\begin{matrix}{\omega = \frac{\sqrt{3}t_{30}}{{3t_{30}^{2}} + \left( {t_{c} + t_{lag}} \right)^{2}}} & (1)\end{matrix}$

In expression (1), t₃₀ denotes a length of time taken by the object tomove from a position corresponding to one of the panning angles θ=+30°and −30° at which the singular point is detected, to a positioncorresponding to the panning angle θ=0°. Furthermore, t_(c) denotes alength of time from a time point when the object passes the positioncorresponding to the panning angle θ=0° to a time point when a last oneof the pre-exposure relative object angular velocities is detected, thatis, to a time point immediately before the release information is input.In addition, t_(lag) denotes a length of time from the time point whenthe last one of the pre-exposure relative object angular velocities isdetected to a midpoint of the exposure time. Although the midpoint ofthe exposure time corresponds to a time point of half of the exposuretime in the following description, the midpoint thereof may be a timepoint other than the point of half thereof as long as the midpoint iswithin the exposure time.

FIG. 10 shows t₃₀, t_(c) and t_(lag) in a case where t₃₀ is a length oftime taken by the object to move from the position of θ=+30° to theposition of θ=0°.

Derivation of expression (1) will be described with reference to FIG. 9.In FIG. 9, denotes the motion velocity of the object (hereinafterreferred to as “an object velocity”), L denotes the shortest distancebetween the motion track of the object (origin A) and the camera 100,and t denotes a length of time taken by the object to move from aposition of θ to the origin A. The angular velocity ω of the object,which is a temporal differentiation of θ, is calculated as follows.

$\omega = \frac{d\; \theta}{dt}$${\tan \mspace{14mu} \theta} = \frac{vt}{L}$$\theta = {\arctan \left( \frac{vt}{L} \right)}$

When u is defined as follows:

${u = \frac{vt}{L}},$

differentiating arctan(u) by u and developing the differentiation resultby u again gives the following expression.

$\begin{matrix}{\frac{d\; \theta}{du} = \frac{1}{1 + u^{2}}} \\{= \frac{L^{2}}{L^{2} + ({vt})^{2}}}\end{matrix}$

Differentiating u by t gives:

${\frac{du}{dt} = \frac{v}{L}},$

andapplying it to a chain rule of differentiation gives the followingexpression.

$\begin{matrix}{\omega = {\frac{d\; \theta}{dt} = {\frac{d\; \theta}{du} \cdot \frac{du}{dt}}}} \\{= \frac{Lv}{L^{2} + ({vt})^{2}}}\end{matrix}$

Calculating L in FIG. 10 gives:

L=√{square root over (3vt ₃₀)}, and

applying this L and t=t_(c)+t_(mg) to the above expression of ω givesexpression (1).

At step S105, the object angular velocity setter 603 determines whetheror not the singular point corresponding to θ=+30° has been detected. Ifthis singular point has been detected, the object angular velocitysetter 603 proceeds to step S106, and otherwise proceeds to step S107.

At step S106, the object angular velocity setter 603 calculates thein-exposure relative object angular velocity ω by using followingexpression (2). Specifically, the object angular velocity setter 603first calculates a difference the (ω_(n)−ω_(n-1)) of pre-exposurerelative object angular velocities calculated at two newest calculationtimes in the pre-exposure angular velocity history. Next, the objectangular velocity setter 603 calculates a relative object angularvelocity (ω_(n)−ω_(n-1))t_(lag)/t_(f) by using the length of timet_(lag) taken by the object to move from the time point when the lastpre-exposure relative object angular velocity is detected to themidpoint of the exposure time. Thereafter, the object angular velocitysetter 603 calculates (sets) the in-exposure relative object angularvelocity ω, which is a final value, by weighting the calculated relativeobject angular velocity (ω_(n)−ω_(n-1))t_(lag)/t_(f) by a weight W of 1or less.

$\begin{matrix}{\omega = \frac{{W\left( {\omega_{n} - \omega_{n - 1}} \right)}t_{lag}}{tf}} & (2)\end{matrix}$

In expression (2), ω_(n) denotes the angular velocity calculated at anewest calculation time of the two newest calculation times, and ω_(n-1)denotes the angular velocity calculated at a one previous calculationtime from the newest calculation time. Furthermore, t_(f) denotes alength of time between the newest calculation time corresponding toω_(n) and the one previous calculation time corresponding to ω_(n-1).

At step S107, the object angular velocity setter 603 determines whetheror not the singular point corresponding to θ=0° has been detected. Ifthis singular point has been detected, the object angular velocitysetter 603 proceeds to step S108, and otherwise proceeds to step S111.

At step S108, the object angular velocity setter 603 determines whetheror not the pre-exposure angular velocity history includes a relativeobject angular velocity that is symmetrical (hereinafter referred to as“a symmetrical history angular velocity”) relative to θ=0° as a centralpoint of symmetry, with a relative object angular velocity at themidpoint of the exposure time. If the angular velocity history includesthe symmetrical history angular velocity, the object angular velocitysetter 603 proceeds to step S109, and otherwise proceeds to step S110.

FIG. 11 shows the symmetrical history angular velocity. A time duringwhich the angular velocity history is accumulated is referred to as “ahistory accumulation time”. A history accumulation time 1 shown in FIG.11 is shorter than t_(lag)+t_(c) on the right side further than theorigin A, so that the angular velocity history does not include thesymmetrical history angular velocity. On the other hand, a historyaccumulation time 2 is longer than t_(lag)+t_(c) on the right sidefurther than the origin A, so that the angular velocity history includesthe symmetrical history angular velocity.

At step S109, the object angular velocity setter 603 sets thesymmetrical history angular velocity as the in-exposure relative objectangular velocity ω.

At step S110, the object angular velocity setter 603 sets thein-exposure relative object angular velocity ω by using expression (2)in which the weight W is 1 or more.

At step S111, the object angular velocity setter 603 determines whetheror not the singular point corresponding to θ=−30° has been detected. Ifthis singular point has been detected, the object angular velocitysetter 603 proceeds to step S112, and otherwise proceeds to step S113.

At step S112, the object angular velocity setter 603 sets thein-exposure relative object angular velocity ω by using expression (2)in which the weight W is 1 or less.

At step S113, the object angular velocity setter 603 sets thein-exposure relative object angular velocity ω by using expression (2)in which the weight W is 1.

This embodiment enables performing the follow shot assist allowing agood follow shot with reduced object image blur even when the angularvelocity of the object measured from the camera 100 changes.

Embodiment 2

Next, description will be made of a camera as an image capturingapparatus that is a second embodiment (Embodiment 2) of the presentinvention. A configuration of the camera of this embodiment is common tothat of the camera 100 of Embodiment 1, and therefore components of thecamera in this embodiment are denoted by the same reference numerals asthose in Embodiment 1.

Embodiment 1 described the case where the object angular velocitycalculator 134 sends the pre-exposure relative object angular velocityand the calculation time at which the calculation thereof was made tothe follow shot controller 132 and where the follow shot controller 132accumulates the sets of the pre-exposure relative object angularvelocity and the calculation time as the pre-exposure angular velocityhistory. Embodiment 2 will describe a case where the object angularvelocity calculator 134 sends to the follow shot controller 132, inaddition to the pre-exposure relative object angular velocity and thecalculation time, an object distance (distance information) at thecalculation time and a change amount of the panning angle θ shown inFIG. 9 from a previous calculation time of the relative object angularvelocity and where the follow shot controller 132 accumulates the setsof the pre-exposure relative object angular velocity, the calculationtime, the object distance and the change amount of the panning angle θas the pre-exposure angular velocity history. The object distance can becalculated from information on, for example, the positions of the zoomlens 103 and the focus lens (not shown) in the image capturing lens unit101. The change amount of the panning angle θ can be calculated byintegrating the angular velocity data. In this embodiment, an autofocusoperation is performed in order to acquire the object distance, andthereby an in-focus state of the image capturing lens unit 101 for amoving object is maintained such that an accurate object distance of themoving object can be calculated. Furthermore, the object distance may beacquired from position information acquired from GPSs provided to boththe object and camera.

FIG. 12 is a flowchart showing an angular velocity setting process(second process) performed by the object angular velocity setter 603.The object angular velocity setter 603 (that is, the cameramicrocomputer 130) executes this process according to part of the followshot assist control program described in Embodiment 1.

At step S601, the object angular velocity setter 603 receiving aninstruction of setting the in-exposure relative object angular velocityfrom the camera microcomputer 130 reads out the pre-exposure angularvelocity history that the object angular velocity setter 603 haspreviously obtained from the object angular velocity calculator 134 andaccumulated.

At step S602, the object angular velocity setter 603 calculates, fromthe multiple sets of the pre-exposure relative object angular velocityand the calculation time contained in the read-out pre-exposure angularvelocity history, an angular acceleration of the object which is atemporal change rate of its angular velocity, and determines from thecalculation result whether or not the object is in uniform linearmotion. Specifically, the object angular velocity setter 603 determineswhether or not a temporal change of the angular acceleration isequivalent to that in the graph of the angular acceleration shown inFIG. 8. If the object is in uniform linear motion, the object angularvelocity setter 603 proceeds to step S603. If the object is not inuniform linear motion, the object angular velocity setter 603 proceedsto step S604.

At step S603, the object angular velocity setter 603 calculates thein-exposure relative object angular velocity ω by using the objectdistances at two newest calculation times and the change amount of thepanning angle θ between these calculation times in the pre-exposureangular velocity history and using following expressions (3) to (8).

$\begin{matrix}{\omega = \frac{Lv}{L^{2} + ({vt})^{2}}} & (3) \\{L = \frac{{mn}\mspace{14mu} \sin \mspace{14mu} {\Delta\theta}}{D}} & (4) \\{v = \frac{D}{t_{f}}} & (5) \\{{D = \sqrt{m^{2} - {2{mn}\mspace{14mu} \cos \mspace{14mu} {\Delta\theta}} + n^{2}}}{t = \left| {\frac{\sqrt{m^{2} - L^{2}}}{v} - t_{lag}} \right|}} & (6) \\{{m < {{n\mspace{14mu} {and}\mspace{14mu} \sqrt{m^{2} - L^{2}}} + \sqrt{n^{2} - L^{2}}} \neq D}{t = {\frac{\sqrt{m^{2} - L^{2}}}{v} + t_{lag}}}{n > {m\mspace{14mu} {or}}}} & (7) \\{{m \leq {{n\mspace{14mu} {and}\mspace{14mu} \sqrt{m^{2} - L^{2}}} + \sqrt{n^{2} - L^{2}}}} = D} & (8)\end{matrix}$

Description will be made of symbols in expressions (3) to (8) by usingFIGS. 13A to 13D. L and v respectively represent, as also shown in FIG.9, the shortest distance from a camera 100′ of this embodiment to themotion track of the object in uniform linear motion (that is, to theorigin A) and the velocity of the object (object velocity) in uniformlinear motion. Furthermore, t represents a length of time from when theobject passes the origin A to the midpoint of the exposure time, mrepresents an object distance at a certain time (that is, thecalculation time at which the pre-exposure relative object angularvelocity is calculated), and n represents an object distance at a timebefore m (that is, a previous calculation time at which the pre-exposurerelative object angular velocity is calculated). Moreover, Δθ representsa change amount of the panning angle θ from a point (hereinafterreferred to as “a first point”) when the object distance is n to a point(hereinafter referred to as “a second point”) when the object distanceis m. D represents a motion distance from the first point to the secondpoint, and t_(f) represents a length of time from the first point to thesecond point. In addition, t_(lag) represents a length of time from thesecond point to the midpoint of the exposure time.

FIG. 13A shows a case where

$\frac{\sqrt{m^{2} - L^{2}}}{v} \geq t_{lag}$

in expression (7), and FIG. 13B shows a case where

$\frac{\sqrt{m^{2} - L^{2}}}{v} < t_{lag}$

in expression (7).

FIG. 13C shows a case where m≤n in expression (8), and FIG. 13D shows acase where m>n in expression (8).

The shortest distance L expressed by expression (4) can be calculated,in all of FIGS. 13A to 13D, from a similarity relation (m sin θ:D=L:n)between a triangle having sides m sin θ and D and a triangle havingsides L and n. The motion distance D expressed by expression (6) can becalculated, in all of FIGS. 13A to 13D, by a Pythagorean theorem in thetriangle having the sides m sin θ and D.

In FIGS. 13A to 13C, the motion distance D can be calculated from:

D ²=(m sin θ)²+(n−m cos θ)².

In FIG. 13D, the motion distance D can be calculated from:

D ²=(m sin θ)²+(m cos θ−n)².

At step S604, the object angular velocity setter 603 sets thein-exposure relative object angular velocity ω by using expression (2)described in Embodiment 1 with a weight W of 1.

This embodiment can calculate the in-exposure relative object angularvelocity ω by acquiring the object distance at arbitrary two time pointsand the change amount Δθ of the panning angle θ therebetween. Thus, thisembodiment enables performing the follow shot assist allowing a goodfollow shot with reduced object image blur even when the angularvelocity of the object measured from the camera 100′ changes.

Embodiment 3

Next, description will be made of a camera as an image capturingapparatus that is a third embodiment (Embodiment 3) of the presentinvention. A configuration of the camera of this embodiment is common tothat of the camera 100 of Embodiment 1, and therefore components of thecamera in this embodiment are denoted by the same reference numerals asthose in Embodiment 1.

Embodiments 1 and 2 described the case where the in-exposure relativeobject angular velocity is calculated only once before the exposure andthe follow shot assist is performed depending on its calculation result.In this case, although a short exposure time causes no problem, a longexposure time may make it impossible to perform a good follow shotbecause of a change of the relative object angular velocity during thatlong exposure time. Thus, this embodiment sequentially calculates(repetitively updates) the in-exposure relative object angular velocityduring the exposure time and controls the shift drive of the shift lens104 depending on a newest calculation result, which makes it possible toperform a good follow shot even when the exposure time is long.

FIG. 14 is a flowchart showing a follow shot assist process performed bythe camera microcomputer 130 in the follow shot assist mode in thisembodiment. The camera microcomputer 130 executes this process accordingto a follow shot assist control program as a computer program. In FIG.14, steps common to those in the flowchart of FIG. 2 in Embodiment 1 areillustrated with the same step numbers as those in FIG. 2, anddescription thereof is omitted.

In this embodiment, the camera microcomputer 130 causes at step S206, asin Embodiment 1, the object angular velocity calculator 134 to calculatethe pre-exposure relative object angular velocity. After this process(first process), the camera microcomputer 130 causes, at next step S701,the object angular velocity setter 603 to hold the pre-exposure relativeobject angular velocity calculated at step S206 and its calculation timeas a pre-exposure angular velocity history. When the in-exposurerelative object angular velocity is calculated by the method describedin Embodiment 2, the camera microcomputer 130 causes the object angularvelocity setter 603 to hold, in addition to the calculated pre-exposurerelative object angular velocity and the calculation time, the objectdistance at the calculation time and the change amount of the panningangle θ from the previous calculation time of the pre-exposure relativeobject angular velocity. At step 702 during the exposure time (duringimage capturing for recording) after the start of the exposure at stepS209, the camera microcomputer 130 causes the object angular velocitysetter 603 to calculate (set) a new in-exposure relative object angularvelocity. This process corresponds to a second process. The cameramicrocomputer 130 repeats this process at step S702 until the exposureis completed at step S211 to repetitively update the in-exposurerelative object angular velocity. Then, the camera microcomputer 130causes at step S210, at each time when the in-exposure relative objectangular velocity is updated at step S702, the follow shot controller 132to control the shift drive of the shift lens 104 depending on theupdated in-exposure relative object angular velocity.

This embodiment repeats the calculation of the in-exposure relativeobject angular velocity also during the exposure time and performs thefollow shot assist based on the newly calculated in-exposure relativeobject angular velocity. That is, this embodiment increases the numberof the in-exposure relative object angular velocities to be calculated(that is, increases the number of times to calculate the predictioninformation) when the exposure time is longer than a predeterminedperiod. Therefore, this embodiment enables performing the follow shotassist allowing a good follow shot with reduced object image blur evenwhen the exposure time is long.

Embodiment 4

Next, with reference to FIG. 15, description will be made of alens-interchangeable camera system as an image capturing apparatus,which is a fourth embodiment (Embodiment 4) of the present invention. Aninterchangeable lens 140 is detachably attachable to alens-interchangeable camera 141. In FIG. 15, components common to thoseof the camera 100 shown in FIG. 3 in Embodiment 1 are denoted by thesame reference numerals as those in Embodiment 1, and descriptionthereof is omitted.

In this embodiment, a camera microcomputer 144 included in the camera141 and a lens microcomputer 142 included in the interchangeable lens140 share the processes performed by the camera microcomputer 130 inEmbodiment 1. The lens microcomputer 142 and the camera microcomputer144 perform serial communication for sending and receiving informationthrough a mount contact 146 provided to the interchangeable lens 140 anda mount contact 147 provided to the camera 141. The camera microcomputer144 includes the shutter controller 133 and the object angular velocitycalculator 134. The lens microcomputer 142 includes the imagestabilization controller 131 and a follow shot controller 132′. Thefollow shot controller 132′ is different from the follow shot controller132 in Embodiment 1 in receiving information through the serialcommunication from the camera microcomputer 144 (object angular velocitycalculator 134). In this embodiment, the lens microcomputer 142 includedin the interchangeable lens 140 corresponds to a controller, and theobject angular velocity calculator 134 as a calculator is included inthe camera microcomputer 144.

FIG. 16 shows a configuration of a shift drive control system providedto the interchangeable lens 140; the system performs a shift drivecontrol of the shift lens 104 in the follow shot assist mode in thisembodiment. In FIG. 16, components common to those shown in FIG. 6 inEmbodiment 1 are denoted by the same reference numerals as those inEmbodiment 1, and description thereof is omitted.

The follow shot controller 132′ includes a camera information acquirer611, an angular velocity data outputter 612, an object angular velocitysetter 613, the second adder 604 and the second integrator 605.

The camera information acquirer 611 acquires, from the cameramicrocomputer 144 via a communication controller 614, the follow shotsetting information and release information described in Embodiment 1.The angular velocity data outputter 612 samples the angular velocitydata described in Embodiment 1 at the predetermined times and sends thesampled angular velocity data to the object angular velocity calculator134 in the camera microcomputer 144 via the communication controller614.

The object angular velocity setter 613 receives the information sentfrom the object angular velocity calculator 134 in the cameramicrocomputer 144 via the communication controller 614. The sentinformation includes a set (or multiple sets) of the pre-exposurerelative object angular velocity calculated by the object angularvelocity calculator 134 before the exposure and a delay time from thecalculation time (acquisition time) of the pre-exposure relative objectangular velocity to a communication time at which this information issent to the object angular velocity setter 613. The object angularvelocity setter 613 converts the received delay time into a lens-sidecalculation time as an internal time of the lens microcomputer 142 andholds (accumulates) information including multiple sets of thecalculation time and the received pre-exposure relative object angularvelocity as a pre-exposure angular velocity history. Then, the objectangular velocity setter 613 sets (estimates) an in-exposure relativeobject angular velocity by using the pre-exposure angular velocityhistory.

FIG. 17 is a flowchart showing a follow shot assist process performed bythe camera microcomputer 144 in the follow shot assist mode. The cameramicrocomputer 144 executes this process according to a camera-sidefollow shot assist control program that is a computer program. In FIG.17, steps common to those in the flowchart of FIG. 2 in Embodiment 1 areillustrated with the same step numbers as those in FIG. 2, anddescription thereof is omitted.

At step S206, the camera microcomputer 144 causes the object angularvelocity calculator 134 to calculate the pre-exposure relative objectangular velocity. After this step (first step), the camera microcomputer144 sends at step S801, to the lens microcomputer 142, the informationincluding the sets of the calculated pre-exposure relative objectangular velocity and the delay time from the calculation time thereof toa current time point that is the communication time.

Thereafter, if the SW2ON is performed at step S208, the cameramicrocomputer 144 sends at step S802 exposure start time information tothe lens microcomputer 142 and performs an exposure of the image sensor112. Furthermore, after the exposure is completed, the cameramicrocomputer 144 sends at step S803 exposure end time information tothe lens microcomputer 142. Then, the camera microcomputer 144 proceedsto step S212.

FIG. 18 is a flowchart showing a follow shot assist process performed bythe lens microcomputer 142 in the follow shot assist mode. The lensmicrocomputer 142 executes this process according to a lens-side followshot assist control program that is a computer program.

At step S901, the lens microcomputer 142 determines whether or nothaving received the information including the sets of the pre-exposurerelative object angular velocity and the delay time from the cameramicrocomputer 144. If having received the information, the lensmicrocomputer 142 proceeds to step S902, and otherwise repeats theprocess at this step.

At step S902, the lens microcomputer 142 holds the received pre-exposurerelative object angular velocities.

Next, at step S903, the lens microcomputer 142 calculates, from thedelay time received at step S901, the lens-side calculation time of thepre-exposure relative object angular velocity and holds the informationincluding the sets of the calculated calculation time and thepre-exposure relative object angular velocity as the pre-exposureangular velocity history.

At step S904, the lens microcomputer 142 sets a relative object angularvelocity (in-exposure relative object angular velocity) at a midpoint ofthe exposure time. The lens microcomputer 142 performs this setting bythe angular velocity setting process (second process) described inEmbodiment 1 by using the flowchart shown in FIG. 1. Alternatively, thelens microcomputer 142 may perform the above setting by receiving theinformation on the object distance and the change amount of the panningangle θ from camera microcomputer 144 and performing the angularvelocity setting process described in Embodiment 2 by using FIG. 12.

Next at step S905, the lens microcomputer 142 determines whether or nothaving received the exposure start time information from the cameramicrocomputer 144. If having received the exposure start timeinformation, the lens microcomputer 142 proceeds to step S906, andotherwise returns to step S901.

At step S906, the lens microcomputer 142 controls the shift drive of theshift lens 104, through the follow shot controller 132, so as to correctthe displacement amount of the object image on the image plane, as atstep S210 shown in FIG. 2 in Embodiment 1. Simultaneously, if thedetermination that the fast panning is being performed is made as atstep S502 shown in FIG. 5 in Embodiment 1, the lens microcomputer 142controls the shift drive of the shift lens 104 so as to correct theimage blur due to the camera shake through the image stabilizationcontroller 131.

Then, at step S907, the lens microcomputer 142 determines whether or nothaving received the exposure end time information from the cameramicrocomputer 144. If having received the exposure end time information,the lens microcomputer 142 returns to step S901, and otherwise returnsto step S906 to continue the follow shot assist process.

This embodiment allows the lens-interchangeable camera system to performa follow shot assist similar to that described in Embodiment 1 or 2.

In a case of allowing the interchangeable lens 140 to perform a followshot assist similar to that described in Embodiment 3, the lensmicrocomputer 142 performs the process at step S904 in FIG. 18 beforestep S906 and returns to step S904 if not having received the exposureend time information at step S907.

This embodiment described the case where the camera microcomputer 144sends to the lens microcomputer 142 the delay time from the calculationtime of the pre-exposure relative object angular velocity to thecommunication time thereof and the lens microcomputer 142 calculates thelens-side calculation time from the delay time. However, a configurationmay be employed which matches the internal time of the lensmicrocomputer 142 and that of the camera microcomputer 144 beforehandand the camera microcomputer 144 sends to the lens microcomputer 142 thepre-exposure relative object angular velocity and the calculation timethereof.

Embodiment 5

Next, description will be made of a lens-interchangeable camera systemas an image capturing apparatus, which is a fifth embodiment (Embodiment5) of the present invention. Components common to those of theinterchangeable lens 140 and the camera 141 shown in FIGS. 15 and 16 inEmbodiment 4 are denoted by the same reference numerals as those inEmbodiment 4, and description thereof is omitted.

Although Embodiment 4 described the case where the lens microcomputer142 calculates the in-exposure relative object angular velocity,Embodiment 5 will describe a case where the camera microcomputer 144calculates the in-exposure relative object angular velocity.Specifically, the camera microcomputer 144 produces a list ofin-exposure relative object angular velocities in which a time lag to astart time of the exposure into consideration and whose numbercorresponds to a number of shift drive times of the shift lens 104. Thecamera microcomputer 144 sends the list to the lens microcomputer 142.

FIG. 19 is a flowchart showing a follow shot assist process performed bythe camera microcomputer 144 in the follow shot assist mode in thisembodiment. The camera microcomputer 144 executes this process accordingto a camera-side follow shot assist control program that is a computerprogram. In FIG. 19, steps common to those in the flowchart of FIG. 17in Embodiment 4 are illustrated with the same step numbers as those inFIG. 17, and description thereof is omitted.

At step S1001, the camera microcomputer 144 holds the pre-exposurerelative object angular velocity calculated at step S206 (firstprocess).

Next at step S1002, the camera microcomputer 144 performs the sameprocess (second process) as that performed at step S207 described inEmbodiment 1 by using FIG. 2 to calculate the in-exposure relativeobject angular velocity. However, at this step, the camera microcomputer144 produces the list including multiple in-exposure relative objectangular velocities for predetermined periodic times of the shift driveof the shift lens 104 and sends this list to the lens microcomputer 142.

For example, in a case where the exposure time is 1/100 seconds and theperiod of the shift drive of the shift lens 104 is 1 kHz, the shift lens104 is driven 10 times within the exposure time. Thus, the cameramicrocomputer 144 calculates the in-exposure relative object angularvelocity for each of the 10 shift drive times and sends the listincluding the 10 calculated in-exposure relative object angularvelocities to the lens microcomputer 142.

In this embodiment, the object angular velocity calculator 134 in thecamera microcomputer 144 calculates the in-exposure relative objectangular velocities and produces the list thereof. That is, the cameramicrocomputer 144 corresponds to a calculator, and the lensmicrocomputer 142 corresponds to a controller.

FIG. 20 is a flowchart showing a follow shot assist process performed bythe lens microcomputer 142 in the follow shot assist mode. The lensmicrocomputer 142 executes this process according to a follow shotassist control program that is a computer program. In FIG. 20, stepscommon to those in the flowchart of FIG. 18 in Embodiment 4 areillustrated with the same step numbers as those in FIG. 18, anddescription thereof is omitted.

At step S1101, the lens microcomputer 142 determines whether or nothaving received the list of the in-exposure relative object angularvelocities from the camera microcomputer 144. If having received thelist, the lens microcomputer 142 proceeds to step S905, and otherwiserepeats the process at this step.

The lens microcomputer 142 having received the exposure start timeinformation from the camera microcomputer 144 at step S905 reads out atstep S1102, from the list received at step S1101, the in-exposurerelative object angular velocities for the respective shift drive timesof the shift lens 104. Then, the lens microcomputer 142 sets thein-exposure relative object angular velocities to be used for therespective shift drive times, in the order from the earliest shift drivetime.

Next at step S1103, the lens microcomputer 142 causes the follow shotcontroller 132′ to control the shift drive of the shift lens 104depending on the in-exposure relative object angular velocity set atstep S1102. In the control of the shift drive, the lens microcomputer142 changes the in-exposure relative object angular velocity used at therespective shift drive times of shift lens 104, in the order set at stepS1102. That is, the in-exposure relative object angular velocity issequentially updated.

Furthermore, in this control of the shift drive, if a determination thatthe fast panning is being performed is made as at step S502 shown inFIG. 5 in Embodiment 1, the lens microcomputer 142 performs the shiftdrive of the shift lens 104 in order to correct the image blur due tothe camera shake through the image stabilization controller 131. Thelens microcomputer 142 repeats the above processes until the exposure iscompleted (step S907). Thereby, this embodiment can respond to thechange of the object angular velocity during the exposure.

Moreover, although the camera and lens microcomputers 144 and 142 sendsand receives the list of the in-exposure relative object angularvelocities as data in this embodiment, sending and receiving a relativeobject angular velocity and a relative object angular acceleration atthe start time of the exposure also enables responding to the change ofthe in-exposure relative object angular velocity. FIG. 21 is a flowchartshowing a follow shot assist process, as a modified example of thisembodiment, performed by the camera microcomputer 144 in the follow shotassist mode. In FIG. 21, steps common to those in the flowchart of FIG.19 are illustrated with the same step numbers as those in FIG. 19, anddescription thereof is omitted.

The camera microcomputer 144 having held at step 1001 the pre-exposurerelative object angular velocity calculated at step S206 calculates atstep S1202 the multiple in-exposure relative object angular velocitiesas at step S1002 in FIG. 19. Then, the camera microcomputer 144calculates, by using the multiple in-exposure relative object angularvelocities, the relative object angular velocity at the start time ofthe exposure and a relative object angular acceleration that is anestimated acceleration as prediction information of the motion of theobject relative to the camera. The camera microcomputer 144 sendsinformation including the relative object angular velocity at the starttime of the exposure and the relative object angular acceleration to thecamera microcomputer 144. Thereafter, the camera microcomputer 144proceeds to step S208 and subsequent steps.

FIG. 22 is a flowchart showing a follow shot assist process performed bythe lens microcomputer 142 in the follow shot assist mode in thismodified example. In FIG. 22, steps common to those in the flowchart ofFIG. 20 are illustrated with the same step numbers as those in FIG. 20,and description thereof is omitted.

At step S1301, the lens microcomputer 142 determines whether or nothaving received the information including the relative object angularvelocity at the start time of the exposure and the relative objectangular acceleration from the camera microcomputer 144. If havingreceived the information, the lens microcomputer 142 proceeds to stepS905, and otherwise repeats the process at this step.

The lens microcomputer 142 determining at step S905 that the exposurehaving been started proceeds to step S1302. At step S1302, the cameramicrocomputer 144 sets, by using the received relative object angularvelocity at the start time of the exposure, the received relative objectangular acceleration and a lapse time after the start time of theexposure, the in-exposure relative object angular velocities for therespective predetermined periodic times (correction times) of the shiftdrive of the shift lens 104.

Then, at step S1103, the lens microcomputer 142 causes, at therespective correction times, the follow shot controller 132′ to controlthe shift drive of the shift lens 104 depending on the in-exposurerelative object angular velocities set at step S1302 for the respectivecorrection times. Thus, the in-exposure relative object angular velocityis sequentially updated.

In this control of the shift drive, if a determination that the fastpanning is being performed is made as at step S502 shown in FIG. 5 inEmbodiment 1, the lens microcomputer 142 performs the shift drive of theshift lens 104 in order to correct the image blur due to the camerashake through the image stabilization controller 131. The lensmicrocomputer 142 repeats the above processes until the exposure iscompleted at step S907. Thereby, this embodiment can respond to thechange of the object angular velocity during the exposure.

This embodiment allows the lens-interchangeable camera system to performa good follow shot with reduced object image blur even when the relativeobject angular velocity changes during the exposure time.

Although each of the above embodiments described the case of performingthe follow shot assist and the image blur correction against the camerashake by shifting the shift lens 104 constituting part of the imagecapturing lens unit 101, the follow shot assist and the image blurcorrection may be performed by shifting the entire image capturing lensunit or by shifting the image sensor 112 as an optical element(shiftable element).

OTHER EMBODIMENTS

Embodiments of the present invention can also be realized by a computerof a system or apparatus that reads out and executes computer executableinstructions recorded on a storage medium (e.g., non-transitorycomputer-readable storage medium) to perform the functions of one ormore of the above-described embodiment(s) of the present invention, andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s). The computer may comprise one or more ofa central processing unit (CPU), micro processing unit (MPU), or othercircuitry, and may include a network of separate computers or separatecomputer processors. The computer executable instructions may beprovided to the computer, for example, from a network or the storagemedium. The storage medium may include, for example, one or more of ahard disk, a random-access memory (RAM), a read only memory (ROM), astorage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application Nos.2015-41436, filed Mar. 3, 2015, and 2015-250675, filed Dec. 22, 2015which are hereby incorporated by reference herein in their entirety.

What is claimed is:
 1. An image capturing apparatus configured toperform image capturing of an object, the apparatus comprising: at leastone processor or at least one circuit for performing the functions of: acontroller configured to control an optical element, when a motion ofthe image capturing apparatus follows a motion of a main object, byusing (a) first motion information obtained from a first detector todetect the motion of the image capturing apparatus and (b) second motioninformation obtained from a second detector to detect a motion of animage of the main object; and a calculator configured to calculateprediction information on a velocity of the image of the main objectduring an exposure time for the image capturing of the main object,which includes a change of the velocity from that before the exposuretime, by using the second motion information detected at differenttimings before the exposure time, wherein the controller is configuredto use the prediction information to control the optical element duringthe exposure time in a follow shot mode to acquire a captured image inwhich the main object is still within the captured image and abackground flows.
 2. An image capturing apparatus according to claim 1,wherein the calculator is configured to calculate the predictioninformation by using: angular velocity information of the image of themain object before the exposure time; angular acceleration informationof the image of the main object before the exposure time; and a starttime at which the exposure time is started.
 3. An image capturingapparatus according to claim 1, wherein the prediction informationincludes a change of a motion from the motion of the image capturingapparatus before the exposure time.
 4. An image capturing apparatusconfigured to perform image capturing of an object, the apparatuscomprising: at least one processor or at least one circuit forperforming the functions of: a controller configured to control anoptical element, by using (a) first motion information obtained from afirst detector to detect a motion of the image capturing apparatus and(b) second motion information obtained from a second detector to detecta motion of an image of a main object; and a calculator configured tocalculate prediction information on the motion of the image of the mainobject during an exposure time for the image capturing of the mainobject, by using the second motion information detected before theexposure time, wherein the controller is configured to use theprediction information to control the optical element during theexposure time in a follow shot mode to acquire a captured image in whichthe main object is still within the captured image and a backgroundflows, and wherein the second motion information includes informationabout acceleration which is calculated by using the information aboutvelocity detected at different timings before the exposure time.
 5. Animage capturing apparatus according to claim 4, wherein the calculatoris configured to calculate the prediction information by using: angularvelocity information of the image of the main object before the exposuretime; angular acceleration information of the image of the main objectbefore the exposure time; and a start time at which the exposure time isstarted.
 6. An image capturing apparatus according to claim 4, whereinthe calculator calculates prediction information on a motion of theimage of the object during an exposure time for the image capturing theobject, by using the first motion information detected during theexposure time and the second motion information detected at differenttimings before the exposure time, wherein the first motion informationincludes information about velocity detected during the exposure time.7. A method of controlling an image capturing apparatus configured toperform image capturing of an object, the method comprising: controllingan optical element, when a motion of the image capturing apparatusfollows a motion of a main object, by using (a) first motion informationobtained from a first detector to detect the motion of the imagecapturing apparatus and (b) second motion information obtained from asecond detector to detect a motion of an image of the main object; andcalculating prediction information on a velocity of the image of themain object during an exposure time for the image capturing of the mainobject, which includes a change of the velocity from that before theexposure time, by using the second motion information detected atdifferent timings before the exposure time, wherein the method controlsthe optical element during the exposure time, by using the predictioninformation in a follow shot mode to acquire a captured image in whichthe main object is still within the captured image and a backgroundflows.
 8. A non-transitory computer-readable storage medium storing acontrol program as a computer program to cause a computer of an imagecapturing apparatus to operate, the image capturing apparatus beingconfigured to perform image capturing of an object, the control programcausing the computer to: control an optical element, when a motion ofthe image capturing apparatus follows a motion of a main object, byusing (a) first motion information obtained from a first detector todetect the motion of the image capturing apparatus and (b) second motioninformation obtained from a second detector to detect a motion of animage of the main object; and calculate prediction information on avelocity of the image of the main object during an exposure time for theimage capturing of the main object, which includes a change of thevelocity from that before the exposure time, by using the second motioninformation detected at different timings before the exposure time,wherein the control program causes the computer to control the opticalelement during the exposure time, by using the prediction information ina follow shot mode to acquire a captured image in which the main objectis still within the captured image and a background flows.
 9. A methodof controlling an image capturing apparatus configured to perform imagecapturing of an object, the method comprising: controlling an opticalelement, by using (a) first motion information obtained from a firstdetector to detect a motion of the image capturing apparatus and (b)second motion information obtained from a second detector to detect amotion of an image of a main object; and calculating predictioninformation on the motion of the image of the main object during anexposure time for the image capturing of the main object, by using thesecond motion information detected before the exposure time, wherein themethod controls the optical element during the exposure time, by usingthe prediction information in a follow shot mode to acquire a capturedimage in which the main object is still within the captured image and abackground flows, and wherein the second motion information includesinformation about acceleration which is calculated by using theinformation about velocity detected at different timings before theexposure time.
 10. A non-transitory computer-readable storage mediumstoring a control program as a computer program to cause a computer ofan image capturing apparatus to operate, the image capturing apparatusbeing configured to perform image capturing of an object, the controlprogram causing the computer to: control an optical element, by using(a) first motion information obtained from a first detector to detect amotion of the image capturing apparatus and (b) second motioninformation obtained from a second detector to detect a motion of animage of a main object; and calculate prediction information on themotion of the image of the main object during an exposure time for theimage capturing of the main object, by using the second motioninformation detected before the exposure time, wherein the controlprogram causes the computer to control the optical element during theexposure time, by using the prediction information in a follow shot modeto acquire a captured image in which the main object is still within thecaptured image and a background flows, and wherein the second motioninformation includes information about acceleration which is calculatedby using the information about velocity detected at different timingsbefore the exposure time.